Video display systems based on spatial light modulators (SLMs) are increasingly being used as an alternative to conventional cathode ray tube (CRT) displays. SLM systems may provide high resolution displays without the bulk and power consumption associated with CRT systems. SLMs, such as digital micromirror devices (DMD) and some plasma and liquid crystal displays, use pulse width modulation (PWM) to create the appearance of intermediate gray-scale intensity levels even though the display device is actually only capable of creating pixels at full intensity. In other words, PWM allows for the recreation of a wide array of gray-scale intensity levels, even though the actual pixels of the display device are only capable of creating either full light or full darkness levels at a particular moment in time.
A digital micromirror device (DMD), for example, is made up of an array of thousands or even millions of bistable mirror elements, interacting with a light source and a projection surface. Each of the mirror elements of the DMD may switch between two positions, corresponding to an open or closed light configuration, based on the angle at which the mirror tilts towards the light source. A micromirror is in an open position when it is oriented to reflect the light source onto the projection surface. A micromirror is in a closed position when it is oriented so that none of the light provided by the light source is projected onto the projection surface. Thus, each micromirror can be oriented in either an open or “on” position, or a closed or “off” position.
By rapidly turning a particular micromirror “on” and “off”, the appropriate intermediate gray-scale intensity level (shade of light) can be projected for a particular pixel on the projection surface. So a “white” pixel may be produced by having the micromirror remain in the open position for the duration of the frame, a “black” pixel may be produced by having the micromirror remain in the closed position for the duration of the frame, and intermediate shades of gray may be produced by switching the micromirror between the open and closed positions over the course of the frame. The gray-scale shade level of the pixel for a given frame would be proportional to the amount of time that the micromirror was “on,” with the gray-scale shade being darker if the “on” time is less than the “off” time, and the gray-scale shade being lighter if the “on” time is greater than the “off” time for a given frame. Color hues may also be added to a DMD projection system by, for example, time multiplexing the white light source through a color wheel and coordinating the switching of each micromirror with respect to the color wheel in order to blend colors to create the desired hue.
In practice, the micromirrors alternate between open and closed positions so fast that the human eye usually cannot discern the discrete “on” and “off” positions of each micromirror. Instead, the human eye extrapolates the discrete binary images projected by each mirror element into a wide variety of pixel shades and hues, integrating the pulses of light in a way that produces a perceived flicker-free brightness level. In this way, DMDs allow for the accurate reproduction of a whole array of shades and hues by taking advantage of the human eye's averaging of quickly varying brightnesses and colors.
PWM typically comprises dividing a frame of incoming video data into weighted bit segments. The weighted bit segments would range in size from the least significant bit (LSB) to the most significant bit (MSB), as shown generally in bit segment 100 in FIG. 1 (showing an exemplary 4-bit binary system). The LSB would typically account for 1/(2n−1) of the time of the refresh period for a frame of video image, where n is the number of bits in the system. Then each succeeding bit typically would be weighted to represent twice the time of its preceding bit. So in a typical binary PWM SLM system, the MSB would last approximately half the refresh period (for a frame), the second most significant bit would last approximately ¼th of the refresh period, the third most significant bit would last approximately ⅛th of the refresh period, and so on until the LSB, which would last 1/(2n−1) of the refresh period.
Each frame, the length of time that an element displays light versus darkness is determined based on the PWM sequence. Typically, each bit segment would correspond to either a “0” representing the “off” state, or a “1” representing the “on” state. Thus, the light displayed will generally depend on which of the weighted bit segments are “on” during the frame. By mixing and matching the bit segments that are “on,” an array of intensity levels may be created. By way of example, FIG. 2 provides a table 200 illustrating the various range of intensity levels that may be reproduced in an exemplary 4-bit binary system. Bit weights are illustrated in the accompanying smaller table 250. If none of the bit segments is “on,” then the mirror displays darkness (i.e., no light is reflected onto the display surface). If all of the bit segments are “on,” then full intensity (maximum available light) would be displayed. A range of intermediate intensities may be created by varying which bit segments are “on” and which bit segments are “off,” with the available intensity levels in the example of FIG. 2 varying in steps of 1/15 the maximum intensity.
Unfortunately, using a basic pulse width modulation scheme of the type described above to create intermediate gray-scale intensity levels can introduce image artifacts. A PWM artifact is a noticeable variation (typically a spike) in the light intensity level, serving as an unwanted blemish on image quality. Such artifacts generally arise because of the segmented, binary nature of SLMs (resulting in discrete, stepped intensity levels rather than a smooth continuum of intensity levels). In certain circumstances, the averaging effect of SLM bit segments may result in unwanted image distortion. An example of such an unwanted PWM artifact phenomenon may be seen in FIG. 3.
In the example of FIG. 3, half of the display screen is set to display light at 7/15 (bit sequence 310, below curve 330) the total available intensity, while the other half of the screen is set to display light at 8/15 (bit sequence 320; above curve 330) the total available intensity. Because of the specific pattern of discrete bit segments associated with these bit sequences and the specific bit segments turned “on” in order to create these two light intensity levels, if the viewer's eye moves from the right side of the screen (showing intensity level 8) to the left side of the screen (showing intensity level 7), there will be a noticeable flaring of light at the transition (rather than the sharply defined step down in intensity sought to be conveyed). As
FIG. 3 illustrates, at the transition the brightness would rise steadily from level 8 to maximum brightness (shown as peak 340), and then drop steadily down to level 7 (instead of merely dropping from level 8 to level 7 along a sharply defined line). This flaring of light as the eye transitions from one intensity level to another (which may degrade the image and noticeably distract the viewer from overall scene continuity) is a good example of a PWM artifact. Such PWM artifacts are a consequence of using modulated light (in discrete segments). These sorts of PWM artifacts are recognized in the art and are generally most visible when there is motion in the image or motion of the viewer's eye (especially when the image includes adjacent image pixels having intensity levels near, and on either side of, the threshold of the most significant intensity bit and/or whenever a new bit is first introduced). Unfortunately, such PWM image artifacts may detract from overall image quality in SLM systems.
In order to try to improve image display quality, a number of techniques have been developed attempting to mitigate PWM artifacts. One of the most effective techniques operates by splitting the duration of the larger bits into multiple, smaller segments and distributing the segments throughout the refresh period. Larger bits, such as the most significant bit, would generally be split into segments that are no smaller than the least significant bit. By dividing and dispersing larger bits throughout the bit sequence, PWM artifacts may be reduced (since bit splitting distributes intensity energy more evenly throughout the refresh period, preventing large spikes in intensity).
FIG. 4 illustrates the manner in which such a bit splitting technique might operate. The example of FIG. 4 shows a typical bit split for intensity levels 7 and 8 in a 4-bit system from the exemplary bit sequence 100 shown in FIG. 1. As can be seen, bit 3 would be divided into four split bits, each the size of bit 1. Similarly, bit 2 is divided into two split bits, each the size of bit 1. Whenever any of the bits would have been “on” in the traditional mode, the associated split bits will be “on” in bit splitting mode. So in FIG. 4, the 0, 1, and 2 bit splits are “on” (while the 3 bit splits are “off”) when creating intensity level 7 (which is 7/15 the total available intensity, shown in new split bit sequence 410), and the 0, 1, and 2 bit splits would be “off” (while the 3 bit splits would be “on”) when creating intensity level 8 (which is 8/15 the total available intensity, shown in new split bit sequence 420).
So rather than utilizing a basic PWM sequence (in which the device is loaded with the MSB and left for approximately ½ the refresh period, then loaded with the second MSB and left for ¼ the refresh period, then loaded with the third MSB and left for ⅛ the refresh period, and so on until the LSB is loaded and left for 1/(2n−1) of the refresh period), an alternative bit splitting technique may be used to reduce PWM artifacts (by spreading intensity energy throughout the bit sequence. Instead of loading and resetting a bit and leaving its micromirror in a single position for the full duration of the bit's allotted time, the longer, more significant bit periods would be broken into smaller split bit segments, which would be distributed throughout the refresh period. The mirror would then be addressed multiple times throughout the refresh period so as to add up to the total bit period duration. Using such a bit splitting technique can create a more pleasing image with less artifacts, serving as an improvement over the more basic PWM sequence (which would leave the mirror in one position for the whole bit period).
FIG. 5 illustrates the mitigation effect that bit splitting may have on PWM artifacts using an exemplary split bit sequence 510. As the example of FIG. 5 shows, splitting the larger bit segments (such as bits 2 and 3 in the example of FIG. 5) acts to divide bits into smaller split bit segments, allowing for distribution of the split bit segments throughout the refresh period and reducing the PWM artifacts experienced by viewers. These splits for the 7/15 and 8/15 intensity levels are shown in split bit sequences 520 and 530, respectively. Using such a bit splitting technique, PWM artifacts at the transition may be reduced to a series of smaller waves (collectively designated 540 on curve 550) which dampen and settle down to the appropriate level, rather than an abrupt spike in intensity level. But while bit splitting may mitigate PWM artifacts, it does not entirely eliminate the problem.
It may be possible to further reduce PWM artifacts, however, by selecting a particular bit sequence whose order minimizes the effect. In non-binary systems, such as that shown in the enumeration table 600 of FIG. 6, there are redundant ways to create some intensity levels. Bit weights are again illustrated in a smaller table 640. For example, in the five bit plane non-binary system (having four effective bits) shown in FIG. 6, there are multiple ways in which to generate intensity levels 3 through 10 (since bits 2, 3, and 4 are all weighted equally, allowing any one of these bits to be substituted for another of the equally weighted bits). So for example, intensity level 4 could be created by turning “on” either bit 2, bit 3, or bit 4. The application engineers typically select which specific bit sequence to use out of the plurality of possible choices for any intensity level, thus creating an enumeration table (like that shown in FIG. 6) listing the specific bit planes to be turned on to create each gray-scale intensity level. For such non-binary systems, designers may use their discretion when selecting which bits to use when creating the enumeration table. Thus, when generating the enumeration table of bit sequences that will be used for various intensity levels in an application, the application engineers may select which of the available bit sequences to use for each intensity level in an effort to minimize artifacts. Similarly, when using bit splitting, there are numerous ways in which to order the split bits (for either binary or non-binary systems) to create a specific intensity level. In other words, regardless of the weighting system used for specific bit segments, the ordering of split bits provides another means for potentially reducing PWM artifacts. So when possible, application engineers generally try to select an appropriate PWM bit sequence for reducing artifacts.
Using human designers to select the bit sequences can be both time and labor intensive. It may take weeks of an application engineer's time and expertise to select an appropriate bit sequence. Unfortunately, attempts to automate the process have conventionally been unsuccessful; computers have typically been unable to effectively select the best bit sequence in a timely manner since the problem may be characterized as NP-hard. In other words, the bit sequence selection process is sufficiently unbounded that there is currently no known way of finding a provably optimal solution without checking every possible solution. But going through every possible variant is a slow and deliberate process, which may significantly delay product development. Thus, there is a need for a computerized technique for quickly optimizing bit sequence selection and reducing PWM artifacts.